Towards Efficient Feedback Control in Streaming Computer Vision Pipelines

Stream processing is currently an active research direction in computer vision. This is due to the existence of many computer vision algorithms that can be expressed as a pipeline of operations, and the increasing demand for online systems that process image and video streams. Recently, a formal stream algebra has been proposed as an abstract framework that mathematically describes computer vision pipelines. The algebra defines a set of concurrent operators that can describe a pipeline of vision tasks, with image and video streams as operands. In this paper, we extend this algebra framework by developing a formal and abstract description of feedback control in computer vision pipelines. Feedback control allows vision pipelines to perform adaptive parameter selection, iterative optimization and performance tuning. We show how our extension can describe feedback control in the vision pipelines of two state-of-the-art techniques.